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A NOVEL USER SENSITIVE INFORMATION ADAPTIVE VIDEO 

TRANSCODING FRAMEWORK 



BACKGROUND 



Background 

[0001] Transcoding refers to the conversion of one digital file to another. 

The conversion includes, but is not limited to, format change, resolution 
change, and bit rate change. In video-on-demand applications, a host 
computer may respond to a user's request to view a stored video file. The host 
computer may transcode the stored video file to an appropriate video format 
and bit rate for transmission through a network to the user. The transcoded 
format may be compatible with the user's platform, e.g., a television or a 
personal computer. The host computer may also adjust the transmission bit 
rate to meet the bandwidth requirement of the network connecting the host 
and the user. 

[0002] Network connection between the host and the user may 

sometimes be unstable or congested. Video transmission on a wireless 
connection such as wireless fidelity (WiFi) network is especially susceptible to 
data loss and errors. Thus, the transcoder on the host usually reduces 
transmission bit rate to protect against such network conditions. However, a 
reduced bit rate typically degrades the quality of the video received by the user. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0003] Embodiments are illustrated by way of example and not by way 

of limitation in the figures of the accompanying drawings in which like 
references indicate similar elements. It should be noted that references to "an" 
or "one" embodiment in this disclosure are not necessarily to the same 
embodiment, and such references mean at least one. 



[0004] FIG. 1 is a block diagram of an embodiment of a video system. 

[0005] FIG. 2 is an example of a frame sequence including three frames. 

[0006] FIG. 3 is a block diagram of a static model of the video system. 

[0007] FIG. 4 is a block diagram of a dynamic model of the video system. 

[0008] FIG. 5 is a flowchart showing operations of a transcoder of the 



video system. 
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DETAILED DESCRIPTION 

[0009] FIG. 1 shows a block diagram of an embodiment of a video 

system 10. Video system 10 may be a personal computer, a file server, or any 
computing device having video transcoding capabilities. In one embodiment, 
video system 10 may be a video-on-demand (VOD) system that transmits a 
video stream to an end user through a network in response to the user's 
request. Video system 10 may be coupled to a memory 12 through a memory 
interface 17 and a memory path 18. Video system 10 may also be coupled to a 
network 15 through a network interface 19 for transmitting video streams to an 
end user. Network 15 may be a wire-lined, wireless network, or a combination 
of both. Network 15 may be a local area network, a wide area network, the 
Internet, or a combination of all of the above. Memory 12 may be a 
combination of one or more of volatile or non-volatile memory devices, or any 
machine-readable medium. For example, a machine-readable medium 
includes read-only memory (ROM); random-access memory (RAM); magnetic 
disk storage media; optical storage media; flash memory devices; biological 
electrical, mechanical systems; electrical, optical, acoustical or other form of 
propagated signals (e.g., carrier waves, infrared signals, or digital signals). 

[0010] Memory 12 may store a plurality of video files, including a media 

stream file 123, in one or more video formats. Media stream file 123 may 
include a sequence of frames. Part of each frame may contain information of 
particular interest or sensitive to a user. For example, FIG. 2 shows three 
consecutive frames, each of which includes a running person and two moving 
cars. The user may be more interested in the person than in the cars, and 
therefore may pay close attention to the details of the person. Thus, the user 
may designate the person as an object containing user sensitive information. 
The areas containing the person, as indicated by ellipses 21-23, are referred to 
as sensitive information parts (SIP) areas. The areas outside of the SIP areas are 
referred to as non-SIP areas. 
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[0011] Referring to FIG. 1, in one embodiment, video system 10 may 

have a transcoding unit 16 comprising transcoder 110, a sensitive information 
parts (SEP) generator 120, and an optional SIP file analyzer 130, for applying a 
biased rate control to the video files. In one embodiment, transcoder 110 may 
assign more bits per macroblock (e.g., a 16-pixel by 16-pixel bock) to the SIP 
area than to the non-SIP area, thereby enhancing the quality of the SIP and the 
user experience. SIP generator 120 generates the SIP information for each 
frame. The SIP information may be generated concurrently with the 
transmission of the transcoded stream, or generated statically into a SIP 
configuration file 125 stored in memory 12. If the SIP information is generated 
offline and stored in SIP configuration file 125, the format of the SIP 
configuration file may not be readily compatible with transcoder 110. SIP file 
analyzer 130 may be used to convert the file format for transcoder 110 to 
resolve any format incompatibility. 

[0012] In FIG- 1, transcoder 110, SIP generator 120, and SIP file analyzer 

130 are shown as hardware devices, which may be implemented by 
Application-Specific Integrated Circuit (ASIC), Field Programmable Gate 
Array (FPGA), or any hardware technology suitable for logic device 
implementation. These hardware devices may have direct access to the files in 
memory 12 through a direct memory access (DMA) controller 13, 
Alternatively, one or more of transcoder 110, SIP generator 120, and SIP file 
analyzer 130 may be implemented as software modules stored in a machine- 
readable medium, which is previously defined. These software modules may 
contain instructions executable by a processor 14. 

[0013] In a static embodiment, the SIP may be generated under the 

directions of a user. For example, a user may manually mark one or more SIP 
areas for each frame and assign each of the marked areas a priority. SIP 
generator 120 may generate the coordinates of each marked area and save 
them in SIP configuration file 125. Alternatively, a user may mark the SEP in 
the frame in which the SIP first appears. SEP generator 120 may use the 
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marked information to automatically locate the SIPs in the frames that follow. 
For example, referring to FIG. 2, a user may manually mark ellipses 21-23 to 
indicate that the running person contains the sensitive information. The user 
may alternatively mark ellipse 21 only. SIP generator 120 may analyze 
characteristics of the object (the running person) or the area contained in ellipse 
21 and search for objects or areas having the same or similar characteristics in 
the succeeding frames. SIP generator 120 may utilize standard functions such 
as those described in the Moving Picture Experts Group-4 (MPEG-4 ) for the 
analysis and search. When SIP generator 120 locates an object or an area, the 
SIP generator may generate a mark, in the shape of an ellipse or any suitable 
shapes, to encircle it. The coordinates of the marks, whether generated by the 
user or SIP generator 120, may be stored in SIP configuration file 125. SIP 
configuration file 125 may store each SIP in the form of an item that includes a 
frame sequence number, a SIP number, a SIP priority, and the shape and 
coordinates of the mark encircling the SIP. 

[0014] The user may alternatively indicate to SIP generator 120 that an 

object (e.g., the running person) is the SIP without encircling the object. In this 
scenario, the user may describe characteristics of the object (e.g., an object of a 
certain color or a certain height-to-width ratio) to SIP generator 120. The user 
may alternatively specify an area of fixed coordinates and shape as the SIP area. 
SIP generator 120 may follow the user's directions to locate the objects or the 
areas in all of the frames. 

[0015] SIP generator 120 may also locate the SIP automatically without 

the directions from a user or with minimal input from a user. For example, a 
user may provide a priority for each of the frequently appearing objects. SIP 
generator 120 may compare the objects in a frame sequence and designate the 
objects that appear the most frequently and /or have the highest priorities as 
the SIP. Alternatively, SIP generator 120 may compare the objects in a 
sequence of frames and designate the objects that appear in the most central 
location of the frames as the SIP. In another scenario, SIP generator 120 may 
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compare the intersected areas in a frame sequence and designate the areas that 
appear the most frequently as the SIP. 

[0016] In some embodiments, video system 10 of FIG. 1 may be 

implemented by a static model or a dynamic model. FIG. 3 and FIG. 4 
illustrate embodiments of a static model and a dynamic model of video system 
10, respectively. In both of the static and dynamic models, transcoder 110 may 
transcode media stream file 123 based on the SIP information and the available 
bandwidth of network 15. Transcoder 110 may determine a different bit rate 
for the SIP and non-SIP areas to ensure that the quality of SIP is not 
compromised by the limited bandwidth. The SIP area may be transmitted at a 
higher bit rate than the non-SIP area. If the available bandwidth is low or if 
network 15 is unstable, transcoder 110 may reduce the bit rate for transmitting 
the non-SIP area but maintain the bit rate for transmitting the SIP area. 
Transcoder 110 may alternatively reduce the bit rates for both the SIP and non- 
SIP areas but apply a higher bit reduction rate to the non-SIP area. To conserve 
more bandwidth for the high-priority SIPs, some of the low-priority SIP areas 
may be discarded. That is, the low-priority SD? areas may be encoded with the 
same bit rate as the non-SIP areas. Thus, video system 10 may adapt to various 
network conditions and utilize the bandwidth efficiently to deliver the 
sensitive information of high quality. 

[0017] In the static model of FIG. 3, SIP configuration file 125 is 

generated prior to the transmission of the transcoded video. SIP configuration 
file 125 may be imported from a different platform and may have a format not 
readily interpretable by transcoder 110. SIP file analyzer 130 may read SIP 
configuration file 125 and convert the Hie format to another format compatible 
with transcoder 110. Transcoder 110 may then generate a transcoded stream 
from media stream file 123 based on the SD? received from SD? file analyzer 130 
and the bandwidth status of network 15. 
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[0018] In the dynamic model of FIG. 4, SIP configuration file 125 and SIP 

file analyzer 130 may be dispensed with. SIP generator 120 generates the SIP 
information concurrently with the transcoding operations and directly sends 
the SIPs to transcoder 110. In one embodiment, transcoder 110 may feed the 
bandwidth status of network 15 to SIP generator 120, allowing the SIP 
generator to dynamically adjust the amount of SIP generated based on the 
network condition. 

[0019] FIG- 5 is a flowchart illustrating an example of the operations of a 

transcoder in some embodiments, e.g., transcoder 110 of FIG. 1. At block 51, 
transcoder 110 receives a bandwidth status indicating the available bandwidth 
for transmitting the transcoded video. At block 52, according to the static 
model of FIG. 3, transcoder 110 receives SIP information from SIP file analyzer 
130. Alternatively, according to the dynamic model of FIG. 4, transcoder 110 
receives SIP information from SIP generator 120 and forwards the bandwidth 
status to the SIP generator. Although the SIP information as shown is received 
after the reception of the bandwidth status, the reception may be in any order 
and may be concurrent. At block 53, based on the bandwidth status, 
transcoder 110 determines the bit rates to transcode the SIP and the non-SIP 
areas. Transcoder 110 may also determine whether to discard the SIP having 
low priority. At block 54, transcoder 110 forms macroblocks approximating 
the marked areas or objects. At block 55, transcoder 110 transcodes each of the 
macroblocks in the SIP areas with a higher bit rate than in the non-SIP areas. 
At block 56, transcoder transmits the transcoded stream to an end user through 
network 15. 

[0020] In the foregoing specification, specific embodiments have been 

described. It will, however, be evident that various modifications and changes 
can be made thereto without departing from the broader spirit and scope of the 
appended claims. The specification and drawings are, accordingly, to be 
regarded in an illustrative rather than a restrictive sense. 
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